Search for: All records

Abstract Decoder-only Transformer models such as Generative Pre-trained Transformers (GPT) have demonstrated exceptional performance in text generation by autoregressively predicting the next token. However, the efficiency of running GPT on current hardware systems is bounded by low compute-to-memory-ratio and high memory access. In this work, we propose a Process-in-memory (PIM) GPT accelerator, PIM-GPT, which achieves end-to-end acceleration of GPT inference with high performance and high energy efficiency. PIM-GPT leverages DRAM-based PIM designs for executing multiply-accumulate (MAC) operations directly in the DRAM chips, eliminating the need to move matrix data off-chip. Non-linear functions and data communication are supported by an application specific integrated chip (ASIC). At the software level, mapping schemes are designed to maximize data locality and computation parallelism. Overall, PIM-GPT achieves 41 − 137 × , 631 − 1074 × speedup and 123 − 383 × , 320 − 602 × energy efficiency over GPU and CPU baseline on 8 GPT models with up to 1.4 billion parameters. 

Genetically encoded fluorescent protein and fluorogenic RNA sensors are indispensable tools for imaging biomolecules in cells. To expand the toolboxes and improve the generalizability and stability of this type of sensor, we report herein a genetically encoded fluorogenic DNA aptamer (GEFDA) sensor by linking a fluorogenic DNA aptamer for dimethylindole red with an ATP aptamer. The design enhances red fluorescence by 4-fold at 650 nm in the presence of ATP. Additionally, upon dimerization, it improves the signal-to-noise ratio by 2–3 folds. We further integrated the design into a plasmid to create a GEFDA sensor for sensing ATP in live bacterial and mammalian cells. This work expanded genetically encoded sensors by employing fluorogenic DNA aptamers, which offer enhanced stability over fluorogenic proteins and RNAs, providing a novel tool for real-time monitoring of an even broader range of small molecular metabolites in biological systems. 

Abstract Reservoir computing (RC) offers efficient temporal data processing with a low training cost by separating recurrent neural networks into a fixed network with recurrent connections and a trainable linear network. The quality of the fixed network, called reservoir, is the most important factor that determines the performance of the RC system. In this paper, we investigate the influence of the hierarchical reservoir structure on the properties of the reservoir and the performance of the RC system. Analogous to deep neural networks, stacking sub-reservoirs in series is an efficient way to enhance the nonlinearity of data transformation to high-dimensional space and expand the diversity of temporal information captured by the reservoir. These deep reservoir systems offer better performance when compared to simply increasing the size of the reservoir or the number of sub-reservoirs. Low frequency components are mainly captured by the sub-reservoirs in later stage of the deep reservoir structure, similar to observations that more abstract information can be extracted by layers in the late stage of deep neural networks. When the total size of the reservoir is fixed, tradeoff between the number of sub-reservoirs and the size of each sub-reservoir needs to be carefully considered, due to the degraded ability of individual sub-reservoirs at small sizes. Improved performance of the deep reservoir structure alleviates the difficulty of implementing the RC system on hardware systems. 

Viral infections are a major global health issue, but no current method allows rapid, direct, and ultrasensitive quantification of intact viruses with the ability to inform infectivity, causing misdiagnoses and spread of the viruses. Here, we report a method for direct detection and differentiation of infectious from noninfectious human adenovirus and SARS-CoV-2, as well as from other virus types, without any sample pretreatment. DNA aptamers are selected from a DNA library to bind intact infectious, but not noninfectious, virus and then incorporated into a solid-state nanopore, which allows strong confinement of the virus to enhance sensitivity down to 1 pfu/ml for human adenovirus and 1 × 10 4 copies/ml for SARS-CoV-2. Applications of the aptamer-nanopore sensors in different types of water samples, saliva, and serum are demonstrated for both enveloped and nonenveloped viruses, making the sensor generally applicable for detecting these and other emerging viruses of environmental and public health concern. 

Abstract The constant drive to achieve higher performance in deep neural networks (DNNs) has led to the proliferation of very large models. Model training, however, requires intensive computation time and energy. Memristor‐based compute‐in‐memory (CIM) modules can perform vector‐matrix multiplication (VMM) in place and in parallel, and have shown great promises in DNN inference applications. However, CIM‐based model training faces challenges due to non‐linear weight updates, device variations, and low‐precision. In this work, a mixed‐precision training scheme is experimentally implemented to mitigate these effects using a bulk‐switching memristor‐based CIM module. Low‐precision CIM modules are used to accelerate the expensive VMM operations, with high‐precision weight updates accumulated in digital units. Memristor devices are only changed when the accumulated weight update value exceeds a pre‐defined threshold. The proposed scheme is implemented with a system‐onchip of fully integrated analog CIM modules and digital sub‐systems, showing fast convergence of LeNet training to 97.73%. The efficacy of training larger models is evaluated using realistic hardware parameters and verifies that CIM modules can enable efficient mix‐precision DNN training with accuracy comparable to full‐precision software‐trained models. Additionally, models trained on chip are inherently robust to hardware variations, allowing direct mapping to CIM inference chips without additional re‐training.